Indigo: User-Level Support for Building Distributed Shared Abstractions

Distributed systems that consist of workstations connected by high performance interconnects ooer computational power comparable to moderate size parallel machines. Middleware like Distributed Shared Memory (DSM) or Distributed Shared Objects (DSO) attempts to improve the programmability of such hardware by presenting to application programmers interfaces similar to those ooered by shared memory machines. This paper presents the portable Indigo communications library which provides a small set of primitives with which arbitrary shared abstractions are easily and eeciently implemented across distributed hardware platforms. Sample shared abstractions implemented with Indigo include DSM and a variety of DSM protocols as well as fragmented objects, where object state is split across diierent machines and where fragment communications may be customized to application-specic consistency needs. The Indigo library's design and implementation are evaluated on two diierent target platforms, a workstation cluster and an IBM SP-2 machine. As part of this evaluation, a novel DSM system and consistency protocol are implemented and evaluated with several high performance applications. Application performance attained with the DSM system is compared to the performance experienced when utilizing the underlying basic message passing facilities or when employing Indigo to construct customized fragmented objects implementing the application's shared state. Such experimentation results in insights concerning the eecient implementation of DSM systems (e.g., how to deal with false sharing). It also leads to the conclusion that Indigo provides a suuciently rich set of abstractions for eecient implementation of the next generation of parallel programming models for high performance machines.